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Introduction 


As  part  of  a  larger  effort  to  develop  a  methodology  for 
evaluating  intelligent  computer  systems,  the  Natural  Language 
Sourcebook  (Read  et  al.,  1990)  was  produced  at  UCLA's  Center  for 
Technology  Assessment.  The  Sourcebook  is  a  compilation  of 
processing  problems  that  intelligent  computer  systems  encounter 
when  faced  with  natural  language  input.  The  problems,  culled  from 
the  literature  in  artificial  intelligence  (AI),  computat ional 
linguistics,  and  cognitive  science,  provide  a  representative 
sample  of  current  processing  issues.  The  entries  in  the 
Sourcebook  are  called  "exemplars."  Each  exemplar  includes  an 
example  of  the  processing  problem  covered,  the  reference  to  the 
literature  source  which  identified  the  problem,  and  a  discussion 
of  the  processing  problem  from  an  AI  system  perspective. 

One  important  use  of  the  Sourcebook  is  as  a  means  for 
evaluating  intelligent  computer  systems  with  regard  to  their 
ability  to  handle  specific  processing  problems.  This  document 
provides  an  empirical  verification  of  the  problem  coverage  in  the 
Natural  Language  Sourcebook  by  referencing  output  from  one 
intelligent  computer  system  to  the  Sourcebook  exemplars .  The  data 
source  for  the  verification  is  a  set  of  163  queries  (questions  and 
commands)  from  a  military  application  (August  1986  CINCPACFLT 
demonstration)  of  a  syntactic  shell  interface,  IRUS,  which  allows 
a  user  access  to  a  database  through  English  (Bates,  Stallard,  and 
Moser,  1985) .  The  set  of  IRUS  queries  is  provided  in  Appendix  A. 

The  IRUS  queries  present  processing  problems  that  span  the 
four  major  categories  of  the  Natural  Language  Sourcebook:  single¬ 
utterance  issues,  connected-utterance  issues,  true-dialogue 
issues,  and  ill-formed  input.  Of  these  four  major  categories,  the 
category  of  single-utterance  issues  is  most  comprehensively 
represented.  This  concentration  of  processing  problems  in  the 
area  of  single-utterance  issues  is  a  result  of  the  nature  of  the 
IRUS  natural  language  system.  Because  IRUS  is  a  query  system 


which  allows  a  user  to  access  a  database  by  means  of  single¬ 
utterance  English  queries,  single-utterance  issues  predominate. 
However,  connected-utterance  and  true-dialogue  issues  arise  when 
several  queries  in  sequence  are  considered  as  a  group. 

This  paper  considers  the  IRUS  queries  in  relation  to  relevant 
Sourcebook  exemplars.  The  discussion  of  processing  problems  in 
the  queries  follows  the  organization  and  sequencing  of  the 
Sourcebook.  Each  discussion  focuses  on  a  single  query  even  when 
there  may  be  several  or  many  queries  that  present  the  same 
processing  problem.  In  cases  where  the  processing  problem  under 
consideration  arises  in  multiple  queries,  the  query  chosen  for 
discussion  is  representative  of  the  larger  group  of  queries.  A 
list  of  queries  organized  into  groups  by  processing  problem 
appears  in  Appendix  B.  In  each  group,  the  query  or  quer'es 
discussed  in  the  text  are  listed  first. 

The  queries  are  considered  as  single  utterances  (Sourcebook 
category  I) ,  as  pairs  of  utterances  presenting  connected-utterance 
processing  problems  (Sourcebook  category  II) ,  as  utterances  in  a 
true-dialogue  context  (Sourcebook  category  III),  and  finally  as 
ill-formed  input  (Sourcebook  category  IV) . 

Natural  Language  Sourcebook  I . 

Single-utterance  issues  are  defined  in  the  Sourcebook  as 
issues  involving  input  which  the  system  can  understand  using  only 
existing,  stored  knowledge.  This  knowledge  specifically  excludes 
knowing  the  results  of  processing  previous  input.  A  typical 
single-utterance  problem  involves  parsing  an  input  in  isolation 
from  other  inputs. 

We  will  begin  by  discussing  IRUS  queries  that  present 
processing  problems  included  in  the  Sourcebook  division  I. A.: 
Single-Utterance  Issues,  the  Identification  of  Syntactic  Units. 
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Natural  Language  Sourcebook  I. A.: 

Single-Utterance  Issues,  Identification  of  Syntactic 
Units . 

Query  40:  What  deployed  ships  are  harpoon  capable 

Left  to  right  processing  of  Query  40  yields  two  potential 
parses  of  the  initial  three  words  "what  deployed  ships":  Wh  Adj 
N  and  Wh  V  N.  Query  40  is  similar  to  the  example  "The  tough 
coach  the  young"  in  Exemplar  I. A.  (#1).  "The  tough  coach"  in 
isolation  can  be  parsed  as  either  Det  Adj  N  or  as  Det  N  V. 
Exemplar  I.  A.  (#1)  discusses  this  issue  of  determining  the 
lexical  categories  of  words.  In  Exemplar  I. A.  (#1)  and  in  Query 
40,  this  problem  could  also  be  described  as  the  problem  of 
recognizing  noun  groups:  "what  deployed  ships"  vs.  "what,"  and 
"the  tough  coach"  vs.  "the  tough." 

Query  77;  What  is  the  personnel  resource  rating  on  Vinson 

Query  77  presents  several  processing  problems.  As  in  Query 
40,  the  identification  of  syntactic  units  as  presented  in  Exemplar 
I. A.  (#1)  is  relevant:  "the  personnel  resource  rating"  can  be 
parsed  as  either  Det  Adj  N  V  or  as  Det  NModifiers  N. 

The  notation  "NModifiers"  to  represent  the  words  "personnel" 
and  "resource"  in  this  latter  parse  subsumes  two  syntactically 
possible  modification  structures .  The  two  possible  expanded 
structures  for  this  parse  are: 

[Det  [  [Adj  [N] ]  N]  ]  and 
[Det  [Adj  [Adj  [N]  ]  ]  ]  . 

Exemplar  I. A.  (#2)  gives  examples  of  both  of  the  above 
modification  structures  and  discusses  principles  to  help  determine 
the  attachment  of  modifiers  to  nouns.  One  principle  involves 
utilizing  semantic  information  about  the  words  involved.  But  in 
Query  77,  everyday  semantic  information  about-  the  words 
"personnel"  and  "resource"  is  not  sufficient  to  decide  between  the 
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two  syntactically  possible  modification  structures.  The  text 
understanding  system  must  have  access  to  additional  semantic 
information:  real-world  knowledge  specific  to  the  military 

context.  The  use  of  real-world  knowledge  in  determining  syntactic 
structure  is  discussed  in  Exemplar  I. A.  (#7). 

Unlike  Query  40,  Query  77  remains  ambiguous  when  the  whole  is 
read:  "rating"  can  remain  either  a  verb  or  a  noun.  When  "rating" 

functions  as  a  verb,  "what"  is  the  object  of  the  verb  "is  rating." 
When  "rating"  functions  as  a  noun,  "what"  is  the  complement  of  the 
copula  "is."  Ambiguity  resulting  from  multiple  parts  of  speech  of 
a  word  (and  the  contingent  structural  ambiguity)  is  discussed  in 
Exemplar  I.B.l.  (#22)  in  regard  to  the  sentence  "I  saw  that 
gasoline  can  explode"  and  in  Exemplar  I. A.  (#1)  in  regard  to  the 
sentence  "The  tough  coach  married  people."  But  the  ambiguity  of 
Query  77  is  specific  to  the  present  participle  form  of  a  verb  and 
question  formation  involving  the  present  participle.  Query  77  and 
similar  queries  (see  Appendix  B)  suggest  an  exemplar  which  would 
specifically  address  ambiguity  involving  the  present  participle. 

Exemplar  I. A.  (#3),  which  discusses  the  identification  of  the 
head  noun  for  alternate  parse  trees,  is  also  relevant  to  Query  77 
except  that  the  ambiguity  of  Query  77  means  that  the  head  noun  of 
the  noun  phrase  "the  personnel.  .  ."  remains  ambiguous  as  well: 
the  head  noun  is  "resource"  if  "rating"  is  parsed  as  a  verb,  and 
it  is  "rating"  if  "rating"  is  parsed  as  a  noun. 

Query  18:  What  ships  deployed  to  the  10  are  C4 

In  addition  to  discussing  the  determination  of  the  head  noun. 
Exemplar  I. A.  (#3)  discusses  the  importance  of  collecting 
potential  parse  trees.  This  aspect  of  Exemplar  I. A.  (#3)  is 
relevant  to  Query  18.  In  left  to  right  sequential  processing, 
"deployed"  can  function  as  either  a  main  verb  or  as  a  past 
participial  adjective.  Until  the  word  "are"  is  read,  it  is  not 
<"’lear  how  "deployed"  is  functioning.  In  either  case,  the  head 
noun  in  Query  18  is  "ships,"  so  the  different  parse  trees  do  not 
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correspond  to  different  head  nouns,  as  they  do  in  Exemplar  I. A. 

(#3)  . 

Natural  Language  Sourcebook 

Single-Utterance  Issues,  Lexical  Ambiguity. 

Query  32:  How  many  subs  in  10  have  SAR  capability 

Query  32  and  many  others  (see  appendix)  make  use  of  the  word 
"have."  "Have"  is  potentially  polysemous,  as  discussed  in 
Exemplar  I.B.l.  (#2),  but  is  used  with  a  single  meaning  in  the 
IRUS  queries:  to  hold  or  maintain  as  a  possession  (e.g.,  harpoon) 
or  characteristic  (e.g.,  INT  readiness  of  M4 ) .  Although  only  one 
meaning  of  "have"  arises  in  the  IRUS  queries.  Exemplar  I.B.l.  (#2) 
is,  nonetheless,  relevant  in  that  a  text  understanding  system 
should  be  prepared  to  handle  more  than  a  single  meaning  of  the 
many  common  meanings  of  a  highly  polysemous  word  like  "have." 

Query  124:  Total  the  C3  submarines 

The  word  "total"  in  Query  124  is  ambiguous.  Exemplar  I.B.l. 
(#19)  raises  the  general  topic  of  lexical  ambiguity:  a  word  can 
have  different  meanings  in  different  syntactic  classes  or  in  the 
same  syntactic  class.  In  isolation,  the  word  "total"  could  be 
either  a  verb  or  a  noun.  In  Query  124,  "total"  is  used  as  a  verb 
and  has  two  possible  meanings  as  a  verb  in  this  query:  "add  up"  or 
"destroy."  Exemplar  I.B.l.  (#16)  discusses  how  disambiguation  can 
be  accomplished  by  considering  the  normal  goals  of  actors.  This 
method  of  disambiguation  does  not  work  for  Query  124  however. 

Both  meanings  of  "total"  are  conceivably  valid  in  the  context  of 
the  goals  of  military  actors  so  additional  context  would  have  to 
be  accessed  in  order  to  resolve  this  ambiguity. 
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Natural  Language  Sourcebook  I.C.I.: 

Single-Utterance  Issues, 

Modifier  Attachment,  Prepositional  Phrase. 

Query  127:  Show  me  the  number  of  submarines  with  readiness  C4 

In  Query  127,  the  prepositional  phrase  "with  readiness  C4" 
could  modify  the  verb  or  any  of  the  noun  phrases  in  this  query. 

For  example.  Query  127  could  be  paraphrased  in  either  of  the 
following  ways:  1)  Show  me,  with  readiness  C3,  the  number  of 
submarines,  or  2)  Show  me  the  number  of  submarines  that  have 
readiness  C3 .  The  problem  of  prepositional  phrase  attachment  is 
presented  generally  in  Exemplar  I.C.l.  (#3).  Potential  readings 
of  the  IRUS  queries  could  be  eliminated  on  the  basis  of  semantic 
world  knowledge:  for  example,  perhaps  readiness  C3  is  a  property 
of  submarines,  not  people.  Exemplar  I.C.l.  (#6)  discusses  how 
proper  attachments  can  be  determined  on  semantic  grounds. 

Query  70:  Display  the  position  of  Kirk 

Query  70  contains  an  "of"  prepositional  phrase,  "of  Kirk, " 
which  modifies  the  noun  "position."  Exemplars  I.C.l.  (#8)  and 
(#9)  discuss  the  attachment  of  "of  phrases  such  as  "of  Kirk." 

The  noun  "position"  expects  an  "of"  phrase  to  follow  (position  of 
what?),  but  not  all  nouns  do  as  is  discussed  in  Exemplar  I.C.l. 

(#9)  . 


6 


Natural  Language  Sourcebook  I.C.2.: 

Single-Utterance  Issues,  Modifier  Attachment,  Other. 

Query  1Q9:  Are  there  ships  in  the  Indian  Ocean  that  are  C4 
Query  17:  What  ships  that  are  C4  are  deployed  in  the  Indian  Ocean 

Query  109  contains  a  relative  clause,  "that  are  C4 . " 

Exemplar  I.C.2.  (#1)  considers  three  problems  in  handling  relative 

clauses  as  postnominal  modifiers.  The  first  of  these  problems, 
finding  the  concept  that  the  clause  is  modifying,  could  be 
relevant  to  query  109  if  subject-verb  agreement  and  semantics  are 
not  considered:  in  this  case  the  relative  clause  "that  are  C4" 
could  modify  either  of  the  nouns  "ships"  or  "Indian  Ocean."  The 
second  topic  of  Exemplar  I.C.2.  (#1),  organizing  the  processing  of 

the  modifier,  applies  to  query  109  in  that  the  relative  clause 
"that  are  C4"  and  the  sentence  "The  ships  are  C4"  should  be 
handled  in  the  same  way.  The  third  issue,  that  of  determining 
where  the  modifying  construction  ends  and  the  higher  level 
construction  continues,  can  be  illustrated  by  comparing  query  109 
with  Query  17.  In  Query  17,  "are  deployed  in  the  Indian  Ocean"  is 
a  continuation  of  the  higher  level  construction,  whereas,  in  Query 
109,  the  sentence  ends  after  the  relative  clause  so  there  is  no 
return  to  the  higher  level  construction. 

Natural  Language  Sourcebook  I.D.4.: 

Single-Utterance  Issues,  Temporal  Reference. 


Query  19:  Did  the  Frederick  deploy  yesterday 

Query  155:  When  will  Vinson  upgrade 

IRUS  Query  19  contains  the  temporal  referent  "yesterday." 
Exemplar  I.D.4.  (#1)  discusses  how  understanding  of  temporal 

referents  involves  manipulation  of  relative  time  intervals.  To 
understand  the  referent  of  "yesterday, "  the  system  must  know  when 
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the  query  was  uttered  and  then  determine  the  referent  of 
"yesterday"  relative  to  this  time.  Query  155  and  others 
containing  the  question  word  "when"  (see  appendix) ,  demand 
responses  that  could  require  an  understanding  of  relative  time  as 
well . 

Natural  Language  Sourcebook  I.F.: 

Single-Utterance  Issues,  Other. 

Query  156:  What  A AN  rating  is  forecasted  for  Vinson 

The  IRUS  queries  include  passive  clauses  such  as  Query  156. 
Passive  clauses  present  different  patterns  of  semantic  roles  than 
do  active  clauses.  The  processing  of  syntax  and  semantics  in 
passive  clauses  is  discussed  in  Exemplar  I.F.  (#2). 

Query  42:  List  ships  which  are  in  the  Indian  Ocean  and  which  are 
TLAM  capable 

In  processing  Query  42,  a  parser  needs  to  determine  what  the 
conjunction  "and"  conjoins  and  what  words,  if  any,  are  missing 
through  ellipsis.  These  two  tasks  are  related:  if  "and"  conjoins 
two  sentences,  the  missing  words  complete  these  sentences,  that 
is,  "List  ships  which  are  in  the  Indian  Ocean  and  [list  ships] 
which  are  TLAM  capable;  if  "and"  conjoins  two  modifying  relative 
clauses,  no  words  are  missing  and  the  sense  of  Query  42  is  to  list 
those  ships  which  are  both  in  the  Indian  Ocean  and  TLAM  capable. 
Exemplars  I.F.  (#8)  and  (#10)  discuss  the  issues  of  coordination 
and  ellipsis,  but  neither  addresses  the  ambiguity  which  can  result 
from  ellipsis.  Query  42  thus  suggests  an  exemplar  which  would 
consider  such  ambiguity. 
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Query  10:  List  the  PACFLT  ships  that  are  C4  or  that  are  C5 

Query  10  illustrates  the  problem  of  ambiguity  resulting  from 
ellipsis  in  a  sentence  containing  the  disjunction  "or."  This 
ambiguity  is  structurally  similar  to  that  in  Query  42  which 
contains  "and"  rather  than  "or."  The  Sourcebook  does  not  contain 
any  examples  with  the  coordinator  "or"  although  it  does  address 
joining  by  "and"  [Exemplars  I.F.  (#8)  and  (#10)].  Query  10  and 
others  (see  Appendix  B)  thus  suggest  an  exemplar  discussing 
coordination  using  "or"  and  ambiguity  resulting  from  coordination 
and  ellipsis  (as  in  Query  42) . 

Query  9:  What's  Hammond's  readiness 

Query  47:  Who  is  the  commanding  officer  of  Vinson 

Many  Wh-questions  such  as  Query  9  and  Query  4  7  are  included 
in  the  IRUS  queries.  There  is  no  Sourcebook  exemplar  which 
discusses  the  processing  of  Wh-questions. 

Wh-questions  correspond  structurally  to  statements,  and  the 
processing  of  a  Wh-question  should  be  similar  to  the  processing  of 
the  corresponding  statement  form.  For  example,  Query  9,  "What's 
Hammond's  readiness,"  is  structurally  similar  to  the  statement 
"Hammmond's  readiness  is  C2,"  and  the  processing  of  question  and 
statement  should  be  similar. 

Natural  Language  Sourcebook  II. 

The  Irus  queries  are  presented  as  single  utterances,  but  when 
utterances  are  used  in  sequence  and  reference  to  previous 
utterances  is  necessary  for  understanding,  connected-utterance 
issues  become  relevant.  As  described  in  the  Sourcebook,  a  natural 
language  understanding  system  need  not  access  any  model  of  the 
user  in  understanding  connected-utterance  issues. 

The  pairs  of  utterances  considered  in  the  following 
discussions  are  not  necessarily  sequentially  presented  in  the 
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source  data  (query  numbers  from  Appendix  A) .  For  the  purposes  of 
the  following  discussions,  it  is  assumed  that  the  paired  queries 
are  adjacent  and  sequential  in  time. 

Natural  Language  Sourcebook  I I. A.: 

Connected-Utterance  Issues,  Anaphora. 

Query  162:  For  what  reason  did  Frederick  downgrade 
Query  163:  When  will  it  change 

Queries  162  and  163  form  a  discourse  which  includes  the 
anaphor  "it"  in  Query  163.  The  word  "it"  has  as  its  antecedent  a 
state  which  is  not  explicitly  mentioned  in  the  text.  The  state  to 
which  "it"  refers,  Frederick's  degraded  status  of  C3,  must  be 
inferred.  Exemplars  II.  A.  (#4)  and  (#12)  discuss  discourse  in 
which  antecedents  must  be  similarly  inferred. 

Query  7:  Display  the  Reeves,  Sterett,  Towers,  Knox,  Cochrane, 

Oldendorf,  Lockwood,  and  Kirk 
Query  134:  Which  of  them  are  C3 

Queries  7  and  134  form  a  discourse  which  includes  the  anaphor 
"them"  in  Query  134 .  The  third  person  plural  pronoun  "them"  does 
not  have  an  explicit  antecedent  in  Query  7;  all  possible 
antecedents  in  Query  7  are  third  person  singular.  An  understander 
must  realize  that  the  singular  ships  in  Query  7  can  be  combined  in 
a  set  and  referred  to  in  the  plural.  Inference  of  a  set  as  an 
antecedent  for  an  anaphor  is  discussed  in  Exemplar  II.  A.  (#15). 
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Query  153:  What  is  the  projected  readiness  of  Vinson 
Query  159:  What  is  the  reason  for  the  degradation 

Queries  153  and  159  form  a  discourse  in  which  the  antecedent 
of  "the  degradation"  must  be  inferred.  This  is  a  complex 
inference  involving  the  recognition  that  the  projected  readiness 
of  Vinson  is  a  state,  comparison  of  two  states  in  time,  and  the 
calculation  that  the  change  of  state  in  time  is  negative. 

Exemplars  II.  A.  (#15),  (#16),  and  (#18)  all  discuss  inference  of 

antecedents  which  are  not  explicitly  mentioned,  but  the  inference 
in  Queries  153  and  159  requires  reasoning  more  complex  than  in 
these  exemplars. 

Query  69:  Where  are  they  now 

The  IRUS  queries  include  many  instances  of  anaphoric 
reference,  some  of  which  are  noted  above.  Simple  anaphoric 
reference  of  pronouns  as  in  Query  69  (the  pronoun  "they")  is 
common.  The  exemplars  involving  anaphora  in  the  Sourcebook  do  not 
discuss  such  simple  anaphoric  reference;  they  are  concerned  with 
more  indirect  and  complex  uses  of  anaphora  such  as  those  discussed 
in  the  preceding  paragraphs . 

Natural  Language  Sourcebook  II.B.: 

Connected-Utterance  Issues,  Ellipsis. 

Query  148:  Which  ships  have  degraded  to  C3 
Query  149:  C4 

Queries  148  and  149  must  be  understood  as  a  pair.  Query  149, 
"C4 , "  is  not  by  itself  a  grammatical  utterance:  it  is  an 
elliptical  phrase  understood  by  analogy  with  query  148.  Exemplars 
II.B.  (#1)  and  (#5)  discuss  similar  pairs  of  utterances. 
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Natural  Language  Sourcebook  III. 


True-dialogue  issues  in  the  Sourcebook  require  that  the 
natural  language  understanding  system  "know  not  only  about  the 
sequence  of  utterances  read,  but  also  something  about  the  user's 
goals,  intents,  and  expectations.  .  .  .  With  true-dialogue 

issues,  the  system  does  not  simply  give  responses  to  the  user's 
input,  but  brings  to  bear  knowledge  about  users  and  their  use  of 
language  in  deciding  on  the  response"  (Read  et  al.,  1990).  If  the 
system  is  intended  to  operate  as  a  knowledgeable  entity,  true- 
dialogue  issues  must  be  addressed. 

Natural  Language  Sourcebook  III.B.: 

True-Dialogue  Issues,  Logical  Presupposition. 

Query  18:  What  ships  deployed  to  the  10  are  C4 

Query  18  presupposes  that  there  exist  ships  deployed  to  the 
Indian  Ocean.  In  order  to  answer  Query  18,  the  system  must  be 
able  not  only  to  process  the  input  utterance,  but  it  must  also 
know  what  the  user's  expectations  in  asking  the  question  are.  If 
there  are  no  ships  deployed  to  the  Indian  Ocean,  it  is  more 
appropriate  for  the  system  to  respond  "There  are  no  ships  deployed 
to  the  10"  than  to  respond  "None"  to  the  query  "What  ships 
deployed  to  the  10  are  C4 . "  Relevant  discussions  of  logical 
presupposition  are  given  in  Exemplars  III.B.  (#1)  and  (#3). 

Natural  Language  Sourcebook  IV. 

The  Sourcebook  divides  ill-formed  input  (Group  IV  of 
exemplars)  into  two  broad  categories:  "input  which  the  user  will 
see  as  ill-formed  (e.g.,  typing  mistakes),  and  input  which  the 
user  expects  the  system  to  handle  but  which  is  nonstandard  m  some 
way"  (Read  et  al.,  1990).  The  IRUS  queries  present  processing 
problems  in  both  of  these  categories. 
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Natural  Language  Sourcebook  IV. A. 1.: 

Ill-Formed  Input,  Mistakes--Miatypinga . 

Query  36:  What  vessels  in  th  Indian  Ocean  have  harpoon 

Query  36  contains  a  misspelled  word.  The  problem  of 
misspelling  is  addressed  in  Exemplar  IV.A.l.  (#1).  In  attempting 
to  match  a  misspelled  word  to  a  set  of  possible  corrections,  the 
parser  must  consider  syntactic  and  semantic  constraints  on  each 
possible  correction. 

Natural  Language  Sourcebook  IV.B.l.: 

Ill-Formed  Input,  Non-Standard  Input--Incomplete  Sentence. 

Query  64:  Diego  Garcia 

In  isolation.  Query  64  can  be  interpreted  as  an  incomplete 
sentence.  Exemplar  IV.B.l.  (#2)  discusses  a  similar  but  not 
totally  parallel  example  of  sentence  fragment.  The  example  in 
Exemplar  IV.B.l.  (#2)  clearly  contains  a  complete  thought  to  a 
native  English  speaker.  Fragments  such  as  Query  64  could 
conceivably  contain  a  complete  thought  to  a  system  user  who 
routinely  uses  shorthand  expressions. 

If  Query  64  occurred  in  context,  perhaps  it  could  be 
understood  as  an  instance  of  ellipsis  [Exemplars  II. B.  (#1)  and 
(#5),  and,  for  example,  IRUS  Queries  148  and  149]. 
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Summary 


From  this  consideration  of  the  IRUS  queries  in  relation  to 
the  Natural  Language  Sourcebook,  it  appears  that  the  coverage  of 
processing  problems  presented  in  the  Sourcebook  is  sufficiently 
comprehensive  to  be  of  practical  use.  Processing  concerns  in  137 
of  the  163  IRUS  queries  relate  to  issues  discussed  in  the 
Sourcebook.  The  remaining  26  queries  (marked  by  an  asterisk  in 
Appendix  B)  fall  into  three  groups:  six  are  Wh-questions  similar 
to  queries  9  and  47  discussed  on  page  11*,  six  include  simple 
anaphoric  reference  as  discussed  on  page  13*,  and  fourteen  are  not 
discussed  in  this  paper.  These  fourteen  queries  (listed  in 
Appendix  B)  are  mainly  simple  imperatives,  e.g..  Query  1,  "Display 
PACOM, "  and  simple  sub ject/auxiliary  inversion  questions,  e.g.. 
Query  61,  "Is  the  Kennedy  in  port." 


*Many  of  the  queries  present  multiple  processing  problems  and 
so  are  listed  in  more  than  one  group  in  Appendix  B.  For  example. 
Query  77  appears  in  the  first  three  groups.  Some  groups,  for 
example  the  group  concerned  with  logical  presupposition  [Exemplars 
III.B.(#1)  and  (#3)]  and  represented  by  Query  18,  are  not  listed 
in  full  because  these  groups  contain  a  very  large  number  of 
queries . 

Multiple  listings  of  queries  means  that  a  query  in  a  group 
not  covered  by  a  Sourcebook  exemplar  may  present  other  processing 
problems  that  are  covered  in  the  Sourcebook.  For  example.  Query 
4,  which  appears  in  a  group  with  no  relevant  exemplar  (page  25), 
also  appears  in  a  group  relevant  to  Exemplars  I.C.l.  (#8)  and  (#9) 
(page  23) . 
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Appendix  A 


Below  is  the  list  of  queries  from  the  August  1986  CINCPACFLT 
demonstration  which  served  as  the  source  of  information  for 
the  linguistic  capabilities  of  IRUS. 


IRUS  QUERIES 

1.  "Display  PACOM" 

2.  "Display  all  carriers" 

3.  "Summarize  the  Midway's  ratings" 

4.  "What  is  the  readiness  of  the  ships  in  Midway's  battle  group" 

5.  "Which  are  LAMPS  capable" 

6.  "Which  are  harpoon  capable?" 

7.  "Display  the  Reeves,  Sterett,  Towers,  Knox,  Cochrane, 
Oldendorf,  Lockwood  and  Kirk?" 

8.  "Where  is  the  Francis  Hammond" 

9-  "What's  Hammond's  readiness" 

10.  "List  the  PACFLT  ships  that  are  C4  or  that  are  CS" 

11.  "Display  the  PACOM  area" 

12.  "Display  all  carriers  in  PACFLT" 

13.  "When  was  the  last  update  to  Midway's  readiness?" 

14.  "Display  positions  of  all  carriers  in  PACFLT" 

15.  "List  the  ships  that  are  c4  or  that  are  c5" 

16.  "List  the  ships  in  PACFLT  that  are  c4  or  that  are  c5" 

17.  "Which  ships  that  are  C4  are  deployed  in  the  Indian  Ocean" 

18.  "What  ships  deployed  to  the  10  are  C4" 

19.  "Did  the  Frederick  deploy  yesterday" 

20.  "List  the  ships  deployed  in  the  Indian  Ocean" 

21.  "What  ships  are  deployed  to  the  Indian  Ocean" 

22 .  "Where  is  the  Carl  Vinson  deployed" 

23.  "Is  the  Frederick  deployed  in  the  Indian  Ocean" 

24 .  "List  the  number  of  ships  that  ARE  deployed  in  INDIAN  OCEAN" 

25.  "List  the  ships  deployed  in  the  Indian  Ocean" 

26.  "What  is  the  total  number  of  destroyers  in  PACFLT" 

27 .  "How  many  cruisers  are  in  EASTPAC" 

28.  "Count  the  frigates  in  WESTPAC" 

29.  "List  the  battleships  in  MIDPAC" 

30.  "What  is  the  total  number  of  cruisers  in  INDIAN  OCEAN" 

31.  "What  ships  in  WESTPAC  are  LINK-11  capable" 

32.  "How  many  subs  in  10  have  SAR  capability" 

33.  "What  ships  are  SAR  capable" 

34  .  "List  THE  harpoon  capable  ships  in  EASTPAC" 

35.  "Does  Frederick  have  TACAN" 

36.  "What  vessels  in  th  INDIAN  OCEAN  have  harpoon" 

37.  "How  many  ships  that  are  harpoon  capable  are  there  in 
the  Pacific  Fleet" 

38.  "How  many  harpoon  capable  ships  are  in  PACFLT" 

39.  "What  ships  in  San  Diego  are  harpoon  capable" 

40.  "What  deployed  ships  are  harpoon  capable" 

41.  "What  deployed  ships  in  the  Indian  Ocean  are  harpoon  capable" 
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42.  "List  ships  which  are  in  the  Indian  Ocean  and  which  are 
TLAM  capable" 

43.  "What  ships  are  TASM  capable" 

44.  "How  many  harpoon  capable  ships  are  there  in  INDIAN  OCEAN?" 

45.  "What's  the  name  of  the  commander  of  Frederick" 

46.  "Who  is  the  CO  of  Frederick" 

47.  "Who  is  the  commanding  officer  of  Vinson" 

48.  "What  Battle  Group  is  Leahy  in" 

49.  "What  are  the  hull  numbers  of  the  ships  with  NTDS" 

50.  "What  is  the  Vinson's  present  speed" 

51.  "What  is  Vinson's  current  course" 

52.  "What's  Vinson's  speed" 

53.  "What  is  the  speed  of  Frederick" 

54.  "What  operational  area  is  the  C4  submarine  in" 

55.  "Which  are  in  SD" 

56.  "How  many  US  ships  are  in  the  INDIAN  OCEAN" 

57.  "List  the  US  Ships" 

58.  "Are  there  any  submarines  in  the  South  China  Sea" 

59.  "What  port  is  that  ship  in" 

60.  "Whut  port  is  that  track  in" 

61.  "Is  the  Kennedy  in  port" 

62.  "List  the  ships  in  San  Diego" 

63.  "List  the  ships  that  are  in  San  Diego" 

64.  "Diego  Garcia" 

65.  "How  many  ships  are  in  San  Diego" 

66.  "Which  port  is  Vinson  in" 

67.  "Is  Vinson  in  San  Diego" 

68.  "What  is  its  position" 

69.  "Where  are  they  now" 

70.  "Display  the  position  of  Kirk" 

71.  "Where  is  the  Carl  Vinson" 

72.  "Which  ships  that  are  C5  are  in  SD?" 

73.  "What  is  the  overall  readiness  of  Frederick" 

74.  "What  is  Frederick's  readiness  today" 

75.  "What  is  the  CROVL  of  Frederick" 

76.  "What  is  Frederick's  combat  rating" 

77.  "What  is  the  personnel  resource  rating  on  Vinson" 

78.  "What  is  its  supplies  resource  readiness" 

79.  "Which  ships  in  WESTPAC  have  an  equipment  resource 
readiness  of  C3" 

80.  "List  the  ships  in  the  INDIAN  OCEAN  with  a  training 
resource  rating  of  C2" 

81.  "What  is  her  AMW  rating" 

82.  "CROVL  Kennedy" 

83.  "CROVL  of  Kennedy" 

84.  "Find  the  overall  readiness  of  John  F.  Kennedy" 

85.  "What  is  Frederick's  current  readiness" 

86.  "What  is  Frederick's  personnel  readiness" 

87.  "Does  Frederick  have  a  personnel  resource  readiness  of  C3" 

88.  "Show  the  current  overall  readiness  of  Vinson" 

89.  "What  is  the  present  AAW  mission  readiness  of  Vinson" 

90.  "What  is  the  ASW  readiness  of  Vinson  right  now" 

91 .  "Show  the  ASU  rating  code  of  Vinson" 

92.  "C3  ships" 
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93.  "What  is  Frederick's  CCC  readiness" 

94.  "Is  Frederick  C4" 

95.  "What  is  her  readiness" 

96.  "Is  Frederick  M4  in  LOG" 

97.  "What  is  the  overall  readiness  of  JL  Brown?" 

98.  "What  is  the  equipment  resource  rating  on  it" 

99.  "What  is  its  supplies  resource  readiness?" 

100.  "Show  the  MOB  rating  code  of  JL  Brown" 

101.  "What  is  its  readiness  today?" 

102.  "What  is  Gallery's  combat  rating?" 

103.  "What  is  Gallery's  MOB  rating?" 

104.  "What  is  CALIFORNIA'S  current  readiness?" 

105.  "Does  INGERSOLL  have  an  INT  readiness  of  M4?" 

106.  "How  many  C-4  ships  are  there" 

107.  "How  many  C-5  ships" 

108.  "Are  there  any  C4  ships  in  the  Indian  Ocean" 

109.  "Are  there  ships  in  the  Indian  Ocean  that  are  C4" 

110.  "What  ships  are  C-5  today" 

111.  "What  are  the  C4  ships" 

112.  "How  many  ships  are  C-5" 

113.  "How  many  of  the  ships  in  INDIAN  OCEAN  are  C-5" 

114.  "How  many  are  C4" 

115.  "Which  ships  are  C4" 

116.  "What  is  the  total  number  of  EASTPAC  ships  that  are  C4" 

117.  "Which  ships  have  a  C4  personnel  resource  rating" 

118.  "How  many  deployed  carriers  are  C3" 

119.  "What  ships  are  reporting  C3" 

120.  "List  the  ships  that  are  reporting  C3" 

121.  "Ships  whose  overall  readiness  is  C3" 

122.  "C3  ships" 

123.  "Count  the  submarines  that  are  C3" 

124  .  "Total  the  C3  submarines" 

125.  "Which  submarines  have  a  readiness  that  is  C3" 

126.  "What  submarines  have  a  C3  readiness  rating" 

127.  "Show  me  the  number  of  submarines  with  readiness  C3" 

128.  "List  the  subs  whose  readiness  is  03” 

129.  "List  the  ships  in  the  INDIAN  OCEAN  with  a  training 
resource  rating  of  C4" 

130.  "How  many  cruisers  are  deployed  that  are  C3" 

131.  "How  many  cruisers  that  are  deployed  are  C3" 

132.  "Which  are  C3" 

133.  "List  those  ships" 

134.  "Which  of  them  are  C3" 

135.  "List  the  C3  ships" 

136.  "How  many  of  them  are  C3" 

137.  "Which  of  these  ships  are  C2" 

138.  "Which  ships  that  are  C4  are  deployed  in  the  Indian  Ocean" 

139.  "What  are  the  C4  deployed  ships  in  the  INDIAN  OCEAN" 

140.  "Which  ships  that  are  C4  are  in  San  Diego" 

141.  "How  many  ships  in  the  INDIAN  OCEAN  are  C-3" 

142.  "List  the  ships  that  are  C4" 

143.  "Which  ships  in  WESTPAC  are  C3?' 

144.  "Which  submarines  in  WESTPAC  have  an  equipment 
resource  readiness  of  C3?" 
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145.  "List  the  ships  whose  overall  readiness  is  C3" 

146.  "How  many  ships  in  the  Third  Fleet  are  C-3" 

147.  "When  was  Frederick’s  readiness  downgraded" 

148.  "Which  ships  have  degraded  to  C3" 

149.  "C4 "  , 

150.  "What  was  their  previous  overall  readiness" 

151.  "What  ships  reported  C3" 

152.  "Did  Frederick  report  a  C3  readiness" 

153.  "What  is  the  projected  readiness  of  Vinson" 

154.  "What  overall  readiness  rating  is  forecasted  for  Vinson" 

155.  "When  will  Vinson  upgrade" 

156.  "What  AAW  rating  is  forecasted  for  Vinson" 

157.  "What  ASU  rating  is  expected  for  Vinson" 

158.  "Does  Frederick  expect  a  C2  readiness" 

159.  "What  is  the  reason  for  the  degradation" 

160.  "Why  did  Frederick  downgrade  to  C3" 

161.  "Why  did  Frederick  downgrade" 

162.  "For  what  reason  did  Frederick  downgrade  to  C3" 

163.  "When  will  it  change" 
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Appendix  B 


IRUS  Queries  Organized  by  Relevant  Exemplars 

(Queries  with  an  underlined  query  number  are  directly  addressed 
in  discussion.  Queries  marked  by  an  asterisk  do  not  present 
processing  problems  covered  in  the  Sourcebook) . 


40 .  What  deployed  ships  are  harpoon  capable. 

41.  What  deployed  ships  in  the  Indian  Ocean  are  harpoon  capable. 
77.  What  is  the  personnel  resource  rating  on  Vinson. 

80.  List  the  ships  in  the  INDIAN  OCEAN  with  a  training  resource 
rating  of  C2 . 

98.  What  is  the  equipment  resource  rating  on  it. 

102.  What  is  Gallery's  combat  rating. 

103.  What  is  Gallery's  MOB  rating. 

117.  Which  ships  have  a  C4  personnel  resource  rating. 

118.  How  many  deployed/carriers/are  C3 . 

129.  List  the  ships  in  the  INDIAN  OCEAN  with  a  training  resource 
rating  of  C4 . 

Exemplar  I.  A.  (#1) 


77 .  What  is  the  personnel  resource  rating  on  Vinson. 

78.  What  is  its  supplies  resource  readiness. 

(Query  78  also  appears  as  query  99  in  Appendix  A) 

89.  What  is  the  present  AAW  mission  readiness  of  Vinson. 
100.  Show  the  MOB  rating  code  of  JL  Brown. 

154.  What  overall  readiness  rating  is  forecasted  for  Vinson. 
Exemplars  I.  A.  (#2)  and  I.  A.  (#7). 


77 .  What  is  the  personnel  resource  rating  on  Vinson. 

98.  What  is  the  equipment  resource  rating  on  it. 

102.  What  is  Gallery's  combat  rating. 

103.  What  is  Gallery's  MOB  rating. 

Exemplar  I.  B.  1.  (#22)  is  close  but  better  to  add  an  exemplar 
specific  to  ambiguity  involving  the  present  participle. 


18 .  What  ships  deployed  to  the  10  are  C4 . 
Exemplar  I.  A.  (#3) 
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32 .  How  many  subs  in  10  have  SAR  capability. 

35.  Does  Frederick,  have  TACAN . 

36.  What  vessels  in  th  INDIAN  OCEAN  have  harpoon. 

79.  Which  ships  in  WESTPAC  have  an  equipment  resource  readiness 
of  C3. 

87.  Does  Frederick  have  a  personnel  resource  readiness  of  C3 . 
105.  Does  INGERSOLL  have  an  INT  readiness  of  M4 . 

117.  Which  ships  have  a  C4  personnel  resource  rating. 

125.  Which  submarines  have  a  readiness  that  is  C3 . 

126.  What  submarines  have  a  C3  readiness  rating. 

144.  Which  submarines  in  WESTPAC  have  an  equipment  resource 
readiness  of  C3 . 

Exemplar  I.  B.  1.  (#2). 


124 .  Total  the  C3  submarines. 
Exemplars  I.  B.  1.  (#16)  and  (#19) . 


127 .  Show  me  the  number  of  submarines  with  readiness  C3 . 

12.  Display  all  carriers  in  PACFLT. 

14.  Display  positions  of  all  carriers  in  PACFLT. 

28.  Count  the  frigates  in  WESTPAC. 

29.  List  the  battleships  in  MIDPAC. 

34.  List  the  harpoon  capable  ships  in  EASTPAC . 

62.  List  the  ships  in  San  Diego. 

80.  List  the  ships  in  the  INDIAN  OCEAN  with  a  training  resource 
rating  of  C2 . 

129.  List  the  ships  in  the  INDIAN  OCEAN  with  a  training  resource 
rat ing  of  C4 . 

Exemplars  I.  C.  1.  (#3)  and  (#6) . 


70 .  Display  the  position  of  Kirk. 

4.  What  is  the  readiness  of  the  ships  in  Midway's  battle  group. 
14.  Display  positions  of  all  carriers  in  PACFLT. 

24.  List  the  number  of  ships  that  arc  deployed  in  Indian  Ocean. 
26.  What  is  the  total  number  of  destroyers  in  PACFLT. 

30.  What  is  the  total  number  of  cruisers  in  INDIAN  OCEAN. 

47.  Who  is  the  commanding  officer  of  Vinson. 

49.  What  are  the  hull  numbers  of  the  ships  with  NTDS . 

84.  Find  the  overall  readiness  of  John  F.  Kennedy. 

88.  Show  the  current  overall  readiness  of  Vinson. 

91.  Show  the  ASU  rating  code  of  Vinson. 

100.  Show  the  MOB  rating  code  of  JL  Brown, 

etc . 

Exemplars  I.  C.  1.  (#8)  and  (#9) 
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109.  Are  there  ships  in  the  Indian  Ocean  that  are  C4 . 

17 .  Which  ships  that  are  C4  are  deployed  in  the  Indian 
Ocean . 

(Query  17  also  appears  as  query  138  in  Appendix  A) 

18 .  What  ships  deployed  to  the  10  are  C4 . 

37 .  How  many  ships  that  are  harpoon  capable  are  there  in  the 
Pacific  Fleet. 

72.  Which  ships  that  are  C5  are  in  SD . 

131.  How  many  cruisers  that  are  deployed  are  C3 . 

140.  Which  ships  that  are  C4  are  in  San  Diego. 

Exemplar  I.  C.  2.  (#1) 


19 .  Did  the  Frederick  deploy  yesterday. 

155 .  When  will  Vinson  upgrade. 

13.  When  was  the  last  update  to  Midway's  readiness. 
69.  Where  are  they  now. 

101.  What  is  its  readiness  today. 

147.  When  was  Frederick's  readiness  downgraded. 

163.  When  will  it  change. 

Exemplar  I.  D.  4.  (#1). 


156 .  What  AAW  rating  is  forecasted  for  Vinson. 

154.  What  overall  readiness  rating  is  forecasted  for  Vinson. 

157.  What  ASU  rating  is  expected  for  Vinson. 

Exemplar  I.  F.  (#2) . 


42 .  List  ships  which  are  in  the  Indian  Ocean  and  which 
are  TLAM  capable. 

Exemplars  I.  F.  (#8)  and  (#10).  Suggests  another  exemplar  which 
addresses  the  ambiguity  in  42 . 

10 .  List  the  PACFLT  ships  that  are  C4  or  that  are  C5 . 

15.  List  the  ships  that  are  C4  or  that  are  C5 . 

16.  List  the  ships  in  PACFLT  that  are  C4  or  that  are  C5. 

Similar  to  Exemplars  I.  F.  (#8)  and  (#10)  but  have  "or"  rather 
than  "and".  Suggests  exemplar  with  "or." 
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*  SL.  What's  Hammond's  readiness. 

47 .  Who  is  the  commanding  officer  of  Vinson. 

4.  What  is  the  readiness  of  the  ships  in  Midway's  battle  group. 

*  5.  Which  are  LAMPS  capable. 

*  6.  Which  are  harpoon  capable. 

*  8.  Where  is  the  Francis  Hammond. 

*  21.  What  ships  are  deployed  to  the  Indian  Ocean. 

*  22.  Where  is  the  Carl  Vinson  deployed, 
etc . 

No  relevant  exemplar. 


162 .  For  what  reason  did  Frederick  downgrade  to  C3 . 

163 .  When  will  it  change. 

Exemplars  II.  A.  (#4)  and  (#12). 


7 .  Display  the  Reeves,  Sterett,  Towers,  Knox,  Cochrane, 
Oldendorf,  Lockwood  a..d  Kirk. 

134 .  Which  of  them  are  C3 . 

Exemplar  II.  A.  (#15) 


153 .  What  is  the  projected  readiness  of  Vinson. 

159 .  What  is  the  reason  for  the  degradation. 

Similar  to  II.  A.  (#15),  (#16)  and  (#18)  but  more  complicated 

inference . 


Simple  Anaphoric  Reference: 

69 .  Where  are  they  now. 

*  68.  What  is  its  position. 

78.  What  is  its  supplies  resource  readiness. 

(Query  78  also  appears  as  query  99  in  Appendix  A) 

*  81.  What  is  her  AMW  rating. 

*  95.  What  is  her  readiness. 

98.  What  is  the  equipment  resource  rating  on  it. 

101.  What  is  its  readiness  today. 

*134.  Which  of  them  are  C3 . 

*136.  How  many  of  them  are  C3 . 

*150.  What  was  their  previous  overall  readiness. 

No  relevant  exemplar. 
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148 .  Which  ships  have  degraded  to  C3 . 

149.  C4 . 

106.  How  many  C4  ships  are  there. 

107.  How  many  C5  ships. 

Exemplars  II.  B.  (#1)  and  (#5) 


18 .  What  ships  deployed  to  the  10  are  C4 . 

20.  List  the  ships  deployed  in  the  Indian  Ocean. 

(Query  20  also  appears  as  query  25  in  Appendix  A) 
36.  What  vessels  in  th  Indian  Ocean  have  harpoon. 

38.  How  many  harpoon  capable  ships  are  in  PACFLT . 

39.  What  ships  in  San  Diego  are  harpoon  capable, 
etc . 

Exemplars  III.  B.  (#1)  and  (#3). 


36 .  What  vessels  in  th  Indian  Ocean  have  harpoon. 
Exemplar  IV.  A.  1.  (#1) 


64 .  Diego  Garcia. 

82.  CROVL  Kennedy. 

83.  CROVL  of  Kennedy. 

92.  C3  ships. 

121.  Ships  whose  overall  readiness  is  C3 . 

122.  C3  ships. 

Exemplar  IV.  B.  1.  (#2) 


Queries  not  discussed: 

*  1.  Display  PACOM. 

*  2.  Display  all  carriers. 

*  3.  Summarize  the  Midway's  ratings. 

*  11.  Display  the  PACOM  area. 

*  57.  List  the  US  ships. 

*61.  Is  the  Kennedy  in  port. 

*67.  Is  Vinson  in  San  Diego. 

*94.  I-s  Frederick  C4 . 

*133.  List  those  ships. 

*152.  Did  Frederick  report  a  C3  readiness. 

*158.  Does  Frederick  expect  a  C2  readiness. 

*160.  Why  did  Frederick  downgrade  to  C3. 

*161.  Why  did  Frederick  downgrade. 

*162.  For  what  reason  did  Frederick  downgrade  to  C3 . 
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